Rolling Upgrade from 0.91

2014-09-25 Thread Noam Cohen
Hey guys - We want to upgrade our Storm cluster from 0.91 to 0.92. I couldn't find any procedure out there on how to perform a rolling upgrade for Storm. Do you know if such a procedure is supported? Could a cluster temporarily operate when some of the worker nodes are in lower versions than

RE: High CPU usage by storm workers

2014-09-25 Thread Simon Cooper
We’ve found the same thing. A significant amount of cpu time is being taken up by unlocking (?) within the disruptor queue. From: Rahul Mittal [mailto:rahulmitta...@gmail.com] Sent: 24 September 2014 16:16 To: user@storm.incubator.apache.org Subject: High CPU usage by storm workers Hi We are

RE: What happens on a batch timeout?

2014-09-25 Thread Simon Cooper
Does anyone have any information that could help with this? I'm baffled and don't understand the behaviour we're seeing - events are being received out of order on a batch replay, the only reason I can think is that tuples are left over from the previous batch in the input queues, but trying to

Re: What happens on a batch timeout?

2014-09-25 Thread Mayur Rustagi
Seems to me, it depends on which spout you are using. If you are using Kafka Transactional Spout then replay is consistent each time. In any other queue, batch may be different. This contains the type of spouts their limitations.

RE: Configuration changes and storm cluster

2014-09-25 Thread Gunderson, Richard-CW
Thank you for the responses guys. I’ve done a little experimenting and discovered restarting the daemons (nimbus, supervisor) is not such a big deal for our topologies. We already have a dedicated zookeeper server in our environment, so we had no need to use the included zk library. It was

Re: Configuration changes and storm cluster

2014-09-25 Thread Nathan Leung
https://github.com/apache/storm/blob/master/conf/defaults.yaml On Thu, Sep 25, 2014 at 12:48 PM, Gunderson, Richard-CW richard.gunder...@bestbuy.com wrote: Thank you for the responses guys. I’ve done a little experimenting and discovered restarting the daemons (nimbus, supervisor) is not

LoggingMetricsConsumer

2014-09-25 Thread Raphael Hsieh
Hi, I've been trying to figure out why registerinfg a LoggingMetricsConsumer isn't working for me. I've been able to figure out that it is indeed running, however the handleDataPoints() function is never called. Can someone explain to me how this class is used by Storm in order to log metrics?

Cassandra bolt

2014-09-25 Thread Strulovitch, Zack
I've updated to 0.9.2 from pre-apache version 0.9.0.1 (which broke my Cassandra bolt implemented using this code : https://github.com/tjake/stormscraperhttps://github.com/tjake/stormscraper ) According to some posts, this is due to netty conflict. Could anyone please suggest me an alternative

Re: Cassandra bolt

2014-09-25 Thread Harsha
did you tried [1]https://github.com/ptgoetz/storm-cassandra. On Thu, Sep 25, 2014, at 11:20 AM, Strulovitch, Zack wrote: I've updated to 0.9.2 from pre-apache version 0.9.0.1 (which broke my Cassandra bolt implemented using this code :

Re: Cassandra bolt

2014-09-25 Thread Robert Lee
If you are looking for CQL, I'd suggest: https://github.com/hmsonline/storm-cassandra-cql On Thu, Sep 25, 2014 at 2:22 PM, Harsha st...@harsha.io wrote: did you tried https://github.com/ptgoetz/storm-cassandra. On Thu, Sep 25, 2014, at 11:20 AM, Strulovitch, Zack wrote: I've updated to

RE: Cassandra bolt

2014-09-25 Thread Strulovitch, Zack
Thank you very much Dan From: Dan DeCapria, CivicScience [dan.decap...@civicscience.com] Sent: Thursday, September 25, 2014 3:08 PM To: user@storm.incubator.apache.org Subject: Re: Cassandra bolt This might be useful to you as well: Netty Issue (I'm using Storm-Kafka and Kafka as well): rm

Re: LoggingMetricsConsumer

2014-09-25 Thread John Reilly
It is called by the MetricsConsumerBolt which is created by storm when a worker is starting up. When you define a metrics consumer, you should see metrics output every 60 seconds. Also, I think the metrics code was only introduced in 0.9.0 so you would need to be running at least that version.