Weird Problem with Storm

Dear storm experts, I am using storm 0.9.2-incubating. I encountered a weird problem when using storm. My topology is: KafkaSpout -> SplitBolt -> StoreBolt I wrote a test program to emit messages every millisecond. The system works fine with small input, but when I send more than 3000 messages t

Re: Tuple acked multiple times

No Storm will not do that. Are you sure you're not assigning the same message id to multiple tuples. On Fri, Oct 10, 2014 at 8:24 AM, Simon Cooper < simon.coo...@featurespace.co.uk> wrote: > We’re seeing a puzzling behaviour where the same tuple is acked several > times at the spout. This is cau

Re: slow topology pull for remote supervisors

On Oct 10, 2014, at 4:11 PM, Harsha wrote: > it could be couple of issues more of speculation at this point > nimbus writes the topology assignments to zookeeper and a watcher > triggers on supervisors to read the assignments > and subsequently download the topology jar from nimbus. so this watc

Re: slow topology pull for remote supervisors

it could be couple of issues more of speculation at this point nimbus writes the topology assignments to zookeeper and a watcher triggers on supervisors to read the assignments and subsequently download the topology jar from nimbus. so this watcher trigger could be faster for the same dc and I am

Re: slow topology pull for remote supervisors

Thanks for the response. On Oct 10, 2014, at 9:10 AM, Harsha wrote: > Jeremy, > I haven't run a cluster across data centers but I feel like > zookeeper might be an issue here since nimbus receives > heartbeats from supervisor and workers via zookeeper. Did you >

Re: Why won't storm restart failing tasks?

Hi Itai, Thanks very much for your response and suggestions. My first naive thought is: why is this so hard? Isn't this the most basic fault tolerance provided by the simplest load balancers (which is essentially what a shuffle grouping is)? Second question is that I see config parameters like t

Re: slow topology pull for remote supervisors

Jeremy, I haven't run a cluster across data centers but I feel like zookeeper might be an issue here since nimbus receives heartbeats from supervisor and workers via zookeeper. Did you looked in zookeeper logs if there is any misbehavior. >From your descrip

Tuple acked multiple times

We're seeing a puzzling behaviour where the same tuple is acked several times at the spout. This is causing problems, as we've assumed each tuple is only acked once. Is it valid for storm to ack the same message id several times, or have we screwed up the topology somewhere? In what situations m

Re: ZeroMQ and Supervisor tools

The latest versions of storm use netty by default (0.9.1-incubating and above, I believe), so you no longer need to install 0mq. At work we use supervisord to monitor storm processes. On Fri, Oct 10, 2014 at 10:46 AM, Nick Katsipoulakis wrote: > Hello, > > I am currently trying to install Storm

ZeroMQ and Supervisor tools

Hello, I am currently trying to install Storm in my local cluster. I see in many tutorials that ZeroMQ is installed also. Do I really need ZeroMQ? If yes, what is its purpose? Also, I see that the Storm Cluster Setup suggests that I should run the nimbus and the supervisors in supervised mode. Do

formatting problems with code samples in Storm docs

Hi, I’ve noticed for a little while that all of the code samples on the Storm docs site are missing line breaks, for instance http://storm.apache.org/documentation/Guaranteeing-message-processing.html Some pages, like this one under "What is Storm’s reliability API?”, also have the three ticks

Re: How can I use a time window with the trident api?

I started a project to do sliding and tumbling windows in Storm. It could be used directly or as an example. http://github.com/calrissian/flowmix On Oct 9, 2014 11:54 PM, "姚驰" wrote: > Hello, I'm trying to use storm to manipulate our monitoring data, but I > don't know how to add a time window s

Re: Why won't storm restart failing tasks?

Hi Marc, (I'm a storm newbie myself). As I understand it your question has two parts: 1. Can Storm detect a Bolt that is "stuck". AFAIK the answer is no. 2. Can Storm do anything about it. AFAIK the answer is no. What I would do in your situation: 1. I would write a delegate bolt that would dele

slow topology pull for remote supervisors

We have a cluster where workers are distributed across multiple data centers. Supervisor nodes in the remote dc (remote meaning the dc where nimbus is not running) pull the topology at a very slow rate. It’s not failing, it’s not retrying, it’s not timing out, it’s just very slow. This seems