Hello, I am using a simple topology with a single spout and a single bolt to test some ideas about automatically scaling the number of worker nodes in a Storm cluster. The bolt is set up to use some processor time and the spout sends out tuples at a rate which is slightly higher than the bolt can process.
The number of emitted tuples is indeed higher than the number of acked tuples and some time later there are some failed tuples. Just what I expected. However, after about 5 to 10 minutes, it seems that nextTuple() isn't called as often as before and the number of emitted tuples drops to a level that the bolt is able to keep up with. Is there some sort of automatic throttling of spouts that I'm not aware of? Best regards, Jan Sipke van der Veen
