Hello, I'm interested in this topic also. Thank you for your answer. I didn't knew about Flux, maybe it could do the job for my case, i'll take a lot at it. I have a also question about performance, I assume that passing through the output collector is faster than Kaka, what do you think ?
Thank you, Crina 2015-11-05 12:36 GMT+01:00 Nathan Leung <[email protected]>: > It's not possible to combine several topologies into one, but it should be > possible to write different tuple sinks such that you can configure each > bolt to write to either the output collector or Kafka. Then it's just a > matter of wiring and configuring your bolts differently. > > You can use something like flux ( > http://storm.apache.org/documentation/flux.html) to change how your bolts > are wired without having to rebuild your jar file every time. > On Nov 5, 2015 4:17 AM, "Irina Alles" <[email protected]> wrote: > >> Hello, >> >> >> >> We are currently studying if storm would be appropriate as a part of our >> monitoring system. >> >> We are measuring sensor data, we need to apply different transformation >> steps and store it somewhere or send it further. >> >> This seems to be a basic use case for storm so far, let’s call this setup >> topology A. >> >> >> We would like to be able to add measured sensors and their transformation >> steps (topology B) dynamically without requiring any system downtime. These >> dynamic additions could happen frequently. It will happen that bolts of >> topology B will require the output of certain bolts in A >> >> Is there a best practice to manage this in storm? >> >> >> >> After a certain runtime of the system we will have to manage several >> topologies and we need to assure the communication between them. >> >> We thought about using Kafka for the communication between topologies, >> but with the growing number of topologies this might not be the best >> approach I suppose (‘best’ means in this case: easy to handle, avoiding >> message overhead). >> >> Maybe it would be better to group some topologies to create a greater >> one? How would one do this in storm (I’ve read about the swap feature, but >> it doesn’t seem to be available yet)? >> >> Is there a better approach? >> >> >> >> Thanks! >> >> Irina >> >
