Hi, I recommend profiling with jvisualvm to see which methods are using the most CPU time. This should provide insight into which methods are causing CPU usage to go up.
--John Sent from my iPhone > On May 14, 2016, at 2:42 PM, Alexandre Wilhelm > <[email protected]> wrote: > > Hi guys, > > I’m a new user of storm and I need help for an issue I have with storm. > > At a glance, I have a topology with three spouts and bolts ; > > - Spout -> get messages from Kafka > - Bolt -> transform the message from Kafka to a well formed son > - Bolt -> send the JSON to a kairosDB server > > All of these bolts/spouts are written in go by using the library > https://github.com/jsgilmore/gostorm > > Unfortunately after using the topology for a while, the CPU usage is just > growing and growing and never goes down. For instance after sending 1 million > messages, the CPU stays at 200% of cpu usage. > > I tried to play with the configuration and get better result ; with the > following configuration : > > conf.put(Config.TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS, 1000); > conf.put(Config.TOPOLOGY_DISRUPTOR_BATCH_SIZE, 10); > conf.put(Config.TOPOLOGY_DISRUPTOR_BATCH_TIMEOUT_MILLIS, 500); > conf.put(Config.TOPOLOGY_DISRUPTOR_WAIT_TIMEOUT_MILLIS, 500); > conf.setMaxSpoutPending(1000000); > conf.put(Config.TASK_HEARTBEAT_FREQUENCY_SECS , 600); > > My CPU usage after 1 million messages will be 50%, but it never goes down... > > What could be the issue(s) and how can I debug that ? I’m pretty new to the > java/clojure world as well... > > Im running on storm 1.0.1 fyi > > Thanks in advance, > -- > Alexandre Wilhelm
