I set up a storm cluster using 0.9.1-incubating, one mater node and two
worknodes. For most of time, the topology works fine, however, from time to
time there is a exception at the master node. When I use storm UI, the browser
shows this page.
org.apache.thrift7.transport.TTransportException: java.net.ConnectException:
Connection refused
at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
at
org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:71)
at backtype.storm.ui.core$main_page.invoke(core.clj:241)
at backtype.storm.ui.core$fn__2044.invoke(core.clj:1016)
at compojure.core$make_route$fn__5693.invoke(core.clj:93)
at compojure.core$if_route$fn__5681.invoke(core.clj:39)
at compojure.core$if_method$fn__5674.invoke(core.clj:24)
at compojure.core$routing$fn__5699.invoke(core.clj:106)
at clojure.core$some.invoke(core.clj:2390)
at compojure.core$routing.doInvoke(core.clj:106)
at clojure.lang.RestFn.applyTo(RestFn.java:139)
at clojure.core$apply.invoke(core.clj:603)
at compojure.core$routes$fn__5703.invoke(core.clj:111)
at ring.middleware.reload$wrap_reload$fn__7318.invoke(reload.clj:14)
at backtype.storm.ui.core$catch_errors$fn__2082.invoke(core.clj:1075)
at
ring.middleware.keyword_params$wrap_keyword_params$fn__6327.invoke(keyword_params.clj:27)
at
ring.middleware.nested_params$wrap_nested_params$fn__6364.invoke(nested_params.clj:65)
at ring.middleware.params$wrap_params$fn__6301.invoke(params.clj:55)
at
ring.middleware.multipart_params$wrap_multipart_params$fn__6390.invoke(multipart_params.clj:103)
at ring.middleware.flash$wrap_flash$fn__6561.invoke(flash.clj:14)
at ring.middleware.session$wrap_session$fn__6552.invoke(session.clj:43)
at ring.middleware.cookies$wrap_cookies$fn__6489.invoke(cookies.clj:160)
at ring.adapter.jetty$proxy_handler$fn__7252.invoke(jetty.clj:16)
at
ring.adapter.jetty.proxy$org.mortbay.jetty.handler.AbstractHandler$0.handle(Unknown
Source)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:519)
at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
... 33 more
And here is the content of drpc.log
2014-06-25 11:03:08 o.a.z.s.NIOServerCnxn [ERROR] Thread Thread[Main
Thread,5,main] died
org.apache.thrift7.transport.TTransportException: Could not create ServerSocket
on address 0.0.0.0/0.0.0.0:3772.
at
org.apache.thrift7.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:89)
~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
at
org.apache.thrift7.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:68)
~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
at
org.apache.thrift7.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:61)
~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
at backtype.storm.daemon.drpc$launch_server_BANG_.invoke(drpc.clj:125)
~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
at backtype.storm.daemon.drpc$_main.invoke(drpc.clj:146)
~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
at clojure.lang.AFn.applyToHelper(AFn.java:159) ~[clojure-1.4.0.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
at backtype.storm.daemon.drpc.main(Unknown Source)
~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
2014-06-25
I find on the web and someone says the reason of
org.apache.thrift7.transport.TTransportException is the storm client cant
connect to the nimbus. But my problem here is I have the topology run for hours
and this exception happened, so I think the connection may not be the reason.
Any one know why?
唐思成
