Hi all,
I'm trying to run a topology in Storm and I am facing some scalability
issues. Specifically, I have a topology where KafkaSpouts read from a
Kafka queue and emit messages to bolts which are connected with each
other through directGrouping. (Each bolt is connected with itself as
well as with each one of the other bolts). Spouts subscribe to bolts
with shuffleGrouping. I observe that when I increase the number of
spouts and bolts proportionally, I don't get the speedup I'm expecting
to. In fact, my topology seems to run slower and for the same amount of
data, it takes more time to complete. For example, when I increase
spouts from 4->8 and bolts from 4->8, it takes longer to process the
same amount of kafka messages.
Any ideas why this is happening? Thanks in advance.
Best,
Dimitris Sarlis
