Hi Snehal, Scraping the UI may not be the best way to go. You may want to give SPM for Storm a try: http://sematext.com/spm . This includes some metrics and screenshots: http://blog.sematext.com/2014/01/30/announcement-apache-storm-monitoring-in-spm/
If you want to continue scraping, then you probably want to check ulimit -a and I'm guessing you'll see 1024 for the max number of open files, which you should be able to check with sysctl. Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Mon, Feb 3, 2014 at 8:37 PM, Snehal <[email protected]> wrote: > Hello, > > We have a 5 machine storm cluster running version 0.8.1. To monitor > the current state of the system we query the storm UI every 10 seconds > and scrape the cluster state. As a result of this change we are seeing > the following exception stack trace in the ui.logs. And also, we see > that a lot of socket connections are opened by the storm UI process > which are not closed. Is this a known issue? > > java.net.SocketException: Too many open files > at java.net.Socket.createImpl(Socket.java:397) > at java.net.Socket.getImpl(Socket.java:460) > at java.net.Socket.setSoLinger(Socket.java:900) > at org.apache.thrift7.transport.TSocket.initSocket(TSocket.java:116) > at org.apache.thrift7.transport.TSocket.<init>(TSocket.java:107) > at org.apache.thrift7.transport.TSocket.<init>(TSocket.java:92) > at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:52) > at backtype.storm.ui.core$main_page.invoke(core.clj:153) > at backtype.storm.ui.core$fn__7296.invoke(core.clj:689) > at compojure.core$if_route$fn__5818.invoke(core.clj:39) > at compojure.core$if_method$fn__5811.invoke(core.clj:24) > at compojure.core$routing$fn__5833.invoke(core.clj:98) > at clojure.core$some.invoke(core.clj:2390) > at compojure.core$routing.doInvoke(core.clj:98) > at clojure.lang.RestFn.applyTo(RestFn.java:139) > at clojure.core$apply.invoke(core.clj:603) > > Thanks, > Snehal >
