Hi all,
I'm fairly new to Storm/Trident and am having some trouble with performance
tweaking a non-local Trident topology. For input, I am using Kafka with one
million Kafka messages pre-loaded into the topic queue. I am forcing my
topology to read from the start of the queue using the Kafka spout's
forceFromStart configuration. I am testing these two topologies: One Kafka
spout with one bolt that counts emitted tuples (using Storm's metrics consumer)
and one bolt for general computation.One Kafka spout with one bolt that counts
emitted tuples (using Storm's metrics consumer) and five bolts for general
computation. The only difference between the first and second topologies are
how many additional bolts I have for general computation. For the first one, I
have measured just under 17000 tuples per second.For the second one, the
measurement is three times slower at just under 5000 tuples per second.
Is this expected behavior? Are there any specific guidelines about running a
topology with a large number of bolts in this manner?I would like to be able to
expand the number of general computation bolts to several hundred if that would
be possible.
Thanks,John
