So the issue i'm having is that when my system is healthy my "spout0" bolt will have around 66ms of latency. Every once in awhile my upstream dependency that pushes data into a Kafka Stream will spike to maybe 10,000ms latency. However, instead of increasing my topology's latency by only 10,000ms mine's shoots up to 100,000ms roughly, and I'm not entirely sure why. I took a look at the Storm UI and it seems that the latency of my system isn't too bad at all, EXCEPT for the spout0 bolt. That guy jumps to around 150ms process latency.
So I'm trying to figure out why this might be. On Wed, Jul 9, 2014 at 1:28 PM, Derek Dagit <[email protected]> wrote: > It should be a windowed average measure of the time between when the > component receives a tuple and when it acks the tuple. > > This can be slower if there is batching, aggregating, or joining happening > (the component must wait for a number of other tuples to arrive before it > can ack). > > On the UI, there are tool tips that explain the measurements. They appear > after hovering over the label. > -- > Derek > > > On 7/9/14, 15:22, Raphael Hsieh wrote: > >> Can somebody explain to me what might cause the spout to have a large >> process latency? Currently my spout0 and $spoutcoord-spout0 have latency's >> higher than I would like. >> I'm consuming data from a Kafka stream. >> >> How is this process latency measured ? >> Is it measuring the amount of time it takes to fill a batch with data and >> send it to the first bolt in the topology? >> >> Thanks >> >> -- Raphael Hsieh Amazon.com Software Development Engineer I (978) 764-9014
