Hello,
We are currently studying if storm would be appropriate as a part of our monitoring system. We are measuring sensor data, we need to apply different transformation steps and store it somewhere or send it further. This seems to be a basic use case for storm so far, let’s call this setup topology A. We would like to be able to add measured sensors and their transformation steps (topology B) dynamically without requiring any system downtime. These dynamic additions could happen frequently. It will happen that bolts of topology B will require the output of certain bolts in A Is there a best practice to manage this in storm? After a certain runtime of the system we will have to manage several topologies and we need to assure the communication between them. We thought about using Kafka for the communication between topologies, but with the growing number of topologies this might not be the best approach I suppose (‘best’ means in this case: easy to handle, avoiding message overhead). Maybe it would be better to group some topologies to create a greater one? How would one do this in storm (I’ve read about the swap feature, but it doesn’t seem to be available yet)? Is there a better approach? Thanks! Irina
