How are you verifying that the tuples are failing? If you're looking at storm UI for the exact counts you may be mislead. Storm samples tuples for at a configurable rate (defaulted to 0.05) and extrapolates the metrics shown in the UI based on that number. For dev or testing purposes you can set the _topology.stats.sample.rate_ to 1 in storm.yaml which will cause storm to compute stats based on every tuple.
On Fri, Apr 15, 2016 at 12:33 PM, <[email protected]> wrote: > Hi all, > > I've recently run into a problem where my topology seems to be losing > tuples after some continuous processing. That is the number of tuples > emitted from one bolt doesn't equal the number of tuples ack'ed for the > downstream bolt. It's also not reporting any tuples as having failed, I ack > immediately in each exectue method, and there seem to be no errors in the > logs. Due to the nature of the topology, one bolt tends to emit about 10 > tuples for each tuple that it receives, resulting in the topology itself > getting backed up relatively quickly. I've read in other articles that this > can result in a memory leak, which might be the cause of my lost tuples. > > My question is what configuration properties of the topology can I change > that would potentially resolve this problem? I currently have my > executor.send.buffer and executor.receive.buffer set at 16384, the > maxSpoutPending at 500000, and the tupleTimeout at 300000, which I thought > would help, but still have not seen any improvement. Or is there something > else that might be causing this problem? > > Thanks > -- Kevin Conaway http://www.linkedin.com/pub/kevin-conaway/7/107/580/ https://github.com/kevinconaway
