OK, thanks. And is there another option if I don't want to go down the fields grouping path?
On Fri, Aug 8, 2014 at 8:52 PM, P. Taylor Goetz <[email protected]> wrote: > The logic for fields grouping is stateless. It does a hash mod on the > field values. Essentially: > > selectedTask = fields.hashCode() % numTasks > > - Taylor > > On Aug 8, 2014, at 1:39 PM, Jonathan Yom-Tov <[email protected]> wrote: > > It's an option but if I understand correctly the field grouping will make > sure each word goes to the same bolt every time, I'm guessing this requires > some sort of central authority to coordinate. How does that happen? > > > On Fri, Aug 8, 2014 at 8:27 PM, Nathan Leung <[email protected]> wrote: > >> Why not use fields grouping like the example so that you don't have to do >> any coordination across bolts? >> On Aug 8, 2014 12:25 PM, "Jonathan Yom-Tov" <[email protected]> wrote: >> >>> I want to implement a topology that is similar to the RollingTopWords >>> topology in the Storm examples >>> <https://github.com/apache/incubator-storm/tree/master/examples/storm-starter>. >>> The idea is to count the frequency of words emitted. Basically, the spouts >>> emit words at random, the first level bolts count the frequency and pass >>> them on. The twist is that I want the bolts to pass on the frequency of a >>> word only if its frequency in one of the bolts exceeded a threshold. So, >>> for example, if the word "Nathan" passed the threshold of 5 occurrences >>> within a time window on one bolt then all bolts would start passing >>> "Nathan"'s frequency onwards. >>> >>> What I thought of doing is having another layer of bolts which would >>> have the list of words which have passed a threshold. They would then >>> receive the words and frequencies from the previous layer of bolts and pass >>> them on only if they appear in the list. Obviously, this list would have to >>> be synchronized across the whole layer of bolts. >>> >>> Is this a good idea? What would be the best way of implementing it? >>> >> > > > -- > Got a cool idea for a web startup? How about I build it for you? Check out > http://chapter64.com/ > > > -- Got a cool idea for a web startup? How about I build it for you? Check out http://chapter64.com/
