Yes, I "discovered" the cause for not able to have multiple UI daemons.As for
why, I am setting up a cluster where all components are installd identically
everywhere except for Nimbus.That is to help automate our deploys and be able
to scale without having to do custom installs/deploys on servers.
From: Parth Brahmbhatt <[email protected]>
To: [email protected]; Jayesh Thakrar <[email protected]>
Cc: Ankit Toshniwal <[email protected]>
Sent: Tuesday, December 23, 2014 2:01 PM
Subject: Re: Can there be more than 1 storm UI daemon/process in a storm
cluster
You can have as many ui hosts as you want , not sure why would you need that.
UI process just uses nimbus clients to communicate with nimbus so as long as
you have the correct storm.yaml it should work.
ThanksParth
On Dec 23, 2014, at 10:08 AM, Jayesh Thakrar <[email protected]> wrote:
Thank you Ankit.
Yes, nimbus is up and running and I could see the UI on one node.Just the
second/subsequent ones were a problem.
I did some more digging/troubleshooting and found my problem.
Apparently I wanted to use a config file (storm.yaml) that was not in the
default location and I had specified via environment variable CONF_DIR.Looking
up the storm python script I found that the correct env. var should be
"STORM_CONF_DIR".I did that and both the supervisor and ui from that node came
up fine.Incidentally for the other node where the UI was coming up fine was
also one of the zookeeper quorum hosts, so it did not have any problems in
connecting to Zookeeper.
Let me know if that reasoning sounds ok.
ThanksJayesh
From: Ankit Toshniwal <[email protected]>
To: [email protected]; Jayesh Thakrar <[email protected]>
Sent: Tuesday, December 23, 2014 11:48 AM
Subject: Re: Can there be more than 1 storm UI daemon/process in a storm
cluster
There can be only 1 Storm UI daemon per cluster from what I know.
Btw is Nimbus daemon process up and running? I have seen this error when storm
UI was up and running, but Nimbus daemon process was not running.
Ankit
On Tue, Dec 23, 2014 at 9:14 AM, Jayesh Thakrar <[email protected]> wrote:
I am a new storm user and trying to set it up on a multi-node cluster.What I am
seeing is that the first UI daemon works fine, but all subsequent ones (each on
a different server/node of the cluster) have erorrs as shown below.The cluster
otherwise seems fine.
Cluster Summary
Topology summary
Supervisor summary
Nimbus Configuration
Internal Server Error
org.apache.thrift7.transport.TTransportException: java.net.ConnectException:
Connection refused
at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
at
org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:75)
at backtype.storm.ui.core$supervisor_summary.invoke(core.clj:501)
at backtype.storm.ui.core$fn__8016.invoke(core.clj:849)
at compojure.core$make_route$fn__6199.invoke(core.clj:93)
at compojure.core$if_route$fn__6187.invoke(core.clj:39)
at compojure.core$if_method$fn__6180.invoke(core.clj:24)
at compojure.core$routing$fn__6205.invoke(core.clj:106)
at clojure.core$some.invoke(core.clj:2443)
at compojure.core$routing.doInvoke(core.clj:106)
at clojure.lang.RestFn.applyTo(RestFn.java:139)
at clojure.core$apply.invoke(core.clj:619)
at compojure.core$routes$fn__6209.invoke(core.clj:111)
at ring.middleware.reload$wrap_reload$fn__6234.invoke(reload.clj:14)
at backtype.storm.ui.core$catch_errors$fn__8059.invoke(core.clj:909)
at
ring.middleware.keyword_params$wrap_keyword_params$fn__6876.invoke(keyword_params.clj:27)
at
ring.middleware.nested_params$wrap_nested_params$fn__6915.invoke(nested_params.clj:65)
at ring.middleware.params$wrap_params$fn__6848.invoke(params.clj:55)
at
ring.middleware.multipart_params$wrap_multipart_params$fn__6943.invoke(multipart_params.clj:103)
at ring.middleware.flash$wrap_flash$fn__7124.invoke(flash.clj:14)
at ring.middleware.session$wrap_session$fn__7113.invoke(session.clj:43)
at ring.middleware.cookies$wrap_cookies$fn__7044.invoke(cookies.clj:160)
at ring.adapter.jetty$proxy_handler$fn__7324.invoke(jetty.clj:16)
at
ring.adapter.jetty.proxy$org.mortbay.jetty.handler.AbstractHandler$0.handle(Unknown
Source)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
... 33 more
Internal Server Error
org.apache.thrift7.transport.TTransportException: java.net.ConnectException:
Connection refused
at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
at
org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:75)
at backtype.storm.ui.core$cluster_configuration.invoke(core.clj:472)
at backtype.storm.ui.core$fn__8012.invoke(core.clj:844)
at compojure.core$make_route$fn__6199.invoke(core.clj:93)
at compojure.core$if_route$fn__6187.invoke(core.clj:39)
at compojure.core$if_method$fn__6180.invoke(core.clj:24)
at compojure.core$routing$fn__6205.invoke(core.clj:106)
at clojure.core$some.invoke(core.clj:2443)
at compojure.core$routing.doInvoke(core.clj:106)
at clojure.lang.RestFn.applyTo(RestFn.java:139)
at clojure.core$apply.invoke(core.clj:619)
at compojure.core$routes$fn__6209.invoke(core.clj:111)
at ring.middleware.reload$wrap_reload$fn__6234.invoke(reload.clj:14)
at backtype.storm.ui.core$catch_errors$fn__8059.invoke(core.clj:909)
at
ring.middleware.keyword_params$wrap_keyword_params$fn__6876.invoke(keyword_params.clj:27)
at
ring.middleware.nested_params$wrap_nested_params$fn__6915.invoke(nested_params.clj:65)
at ring.middleware.params$wrap_params$fn__6848.invoke(params.clj:55)
at
ring.middleware.multipart_params$wrap_multipart_params$fn__6943.invoke(multipart_params.clj:103)
at ring.middleware.flash$wrap_flash$fn__7124.invoke(flash.clj:14)
at ring.middleware.session$wrap_session$fn__7113.invoke(session.clj:43)
at ring.middleware.cookies$wrap_cookies$fn__7044.invoke(cookies.clj:160)
at ring.adapter.jetty$proxy_handler$fn__7324.invoke(jetty.clj:16)
at
ring.adapter.jetty.proxy$org.mortbay.jetty.handler.AbstractHandler$0.handle(Unknown
Source)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
... 33 more
Internal Server Error
org.apache.thrift7.transport.TTransportException: java.net.ConnectException:
Connection refused
at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
at
org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:75)
at backtype.storm.ui.core$all_topologies_summary.invoke(core.clj:515)
at backtype.storm.ui.core$fn__8018.invoke(core.clj:851)
at compojure.core$make_route$fn__6199.invoke(core.clj:93)
at compojure.core$if_route$fn__6187.invoke(core.clj:39)
at compojure.core$if_method$fn__6180.invoke(core.clj:24)
at compojure.core$routing$fn__6205.invoke(core.clj:106)
at clojure.core$some.invoke(core.clj:2443)
at compojure.core$routing.doInvoke(core.clj:106)
at clojure.lang.RestFn.applyTo(RestFn.java:139)
at clojure.core$apply.invoke(core.clj:619)
at compojure.core$routes$fn__6209.invoke(core.clj:111)
at ring.middleware.reload$wrap_reload$fn__6234.invoke(reload.clj:14)
at backtype.storm.ui.core$catch_errors$fn__8059.invoke(core.clj:909)
at
ring.middleware.keyword_params$wrap_keyword_params$fn__6876.invoke(keyword_params.clj:27)
at
ring.middleware.nested_params$wrap_nested_params$fn__6915.invoke(nested_params.clj:65)
at ring.middleware.params$wrap_params$fn__6848.invoke(params.clj:55)
at
ring.middleware.multipart_params$wrap_multipart_params$fn__6943.invoke(multipart_params.clj:103)
at ring.middleware.flash$wrap_flash$fn__7124.invoke(flash.clj:14)
at ring.middleware.session$wrap_session$fn__7113.invoke(session.clj:43)
at ring.middleware.cookies$wrap_cookies$fn__7044.invoke(cookies.clj:160)
at ring.adapter.jetty$proxy_handler$fn__7324.invoke(jetty.clj:16)
at
ring.adapter.jetty.proxy$org.mortbay.jetty.handler.AbstractHandler$0.handle(Unknown
Source)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
... 33 more
Internal Server Error
org.apache.thrift7.transport.TTransportException: java.net.ConnectException:
Connection refused
at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
at
org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:75)
at backtype.storm.ui.core$cluster_summary.invoke(core.clj:477)
at backtype.storm.ui.core$fn__8014.invoke(core.clj:847)
at compojure.core$make_route$fn__6199.invoke(core.clj:93)
at compojure.core$if_route$fn__6187.invoke(core.clj:39)
at compojure.core$if_method$fn__6180.invoke(core.clj:24)
at compojure.core$routing$fn__6205.invoke(core.clj:106)
at clojure.core$some.invoke(core.clj:2443)
at compojure.core$routing.doInvoke(core.clj:106)
at clojure.lang.RestFn.applyTo(RestFn.java:139)
at clojure.core$apply.invoke(core.clj:619)
at compojure.core$routes$fn__6209.invoke(core.clj:111)
at ring.middleware.reload$wrap_reload$fn__6234.invoke(reload.clj:14)
at backtype.storm.ui.core$catch_errors$fn__8059.invoke(core.clj:909)
at
ring.middleware.keyword_params$wrap_keyword_params$fn__6876.invoke(keyword_params.clj:27)
at
ring.middleware.nested_params$wrap_nested_params$fn__6915.invoke(nested_params.clj:65)
at ring.middleware.params$wrap_params$fn__6848.invoke(params.clj:55)
at
ring.middleware.multipart_params$wrap_multipart_params$fn__6943.invoke(multipart_params.clj:103)
at ring.middleware.flash$wrap_flash$fn__7124.invoke(flash.clj:14)
at ring.middleware.session$wrap_session$fn__7113.invoke(session.clj:43)
at ring.middleware.cookies$wrap_cookies$fn__7044.invoke(cookies.clj:160)
at ring.adapter.jetty$proxy_handler$fn__7324.invoke(jetty.clj:16)
at
ring.adapter.jetty.proxy$org.mortbay.jetty.handler.AbstractHandler$0.handle(Unknown
Source)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
... 33 more
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader of
this message is not the intended recipient, you are hereby notified that any
printing, copying, dissemination, distribution, disclosure or forwarding of
this communication is strictly prohibited. If you have received this
communication in error, please contact the sender immediately and delete it
from your system. Thank You.
