storm only allows static topologies. however you could always create a fully connected graph, and then implement CustomStreamGrouping with your own dynamic logic.
On Sat, Apr 12, 2014 at 2:45 AM, M.Tarkeshwar Rao <[email protected]>wrote: > Hi All, > > Any one have idea about how reportError is working. > > Can you please share some links for storm internals and how ackers are > working. > > Regards > > > On Sat, Apr 12, 2014 at 6:44 AM, Srinath C <[email protected]> wrote: > >> Hi Otis, >> The type of spouts and bolts and the inter-connectivity between them >> is fixed. The number of (tasks) spouts and bolts is configurable but is >> also fixed for the lifetime of the topology. The topology should be >> re-deployed if it changes. However, the number of threads that work with >> these tasks are updateable by deactivating and re-activating the topology. >> >> But, I did feel the need to dynamically add/remove bolts. In our >> environment, users configure policies which do specific aggregations over >> the data streams. Since these policies would be created/deleted by user, I >> thought it would be great to be able to dynamically alter the topology. >> Since that is not possible using Storm, we had to circumvent it by >> re-designing our streams. >> >> Regards, >> Srinath. >> >> >> >> >> On Fri, Apr 11, 2014 at 6:41 PM, Otis Gospodnetic < >> [email protected]> wrote: >> >>> Hi, >>> >>> I'm wondering how dynamic topologies are for most Storm users? >>> That is, do people tend to have topologies with a more or less fixed >>> type (and number?) of spouts & bolts? Or do people have topologies where >>> different types of spouts & bolts are regularly being added/removed? >>> >>> I'm asking because some of our SPM users who use it to monitor Storm >>> performance metrics are asking if they can see metrics for their spouts and >>> bolts. We didn't collect these metrics in the SPM based on the assumption >>> that topologies are very dynamic, meaning that spouts and bolts change a >>> lot and one would end up with a messy pile of different kinds of bolts & >>> spouts in the SPM UI, but..... >>> >>> .... now I'm wondering, is this actually a correct assumption? I now >>> get the feeling most people have 1 or more topologies with a more or less >>> fixed types (and even number?) of spouts & bolts. >>> >>> Could anyone please enlighten me? >>> >>> Thanks, >>> Otis >>> -- >>> Performance Monitoring * Log Analytics * Search Analytics >>> Solr & Elasticsearch Support * http://sematext.com/ >>> >>> >> >
