1 second is too short. Spout latency includes time spent in the output queue from the spout (increasing max spout pending potentially increases your end-to-end latency, depending on whether you have anything buffered in the spout output queues).
On Wed, Jul 29, 2015 at 12:40 PM, Kashyap Mhaisekar <[email protected]> wrote: > Thanks Nathan. But in this case how should the Spout Latency be > interpreted. In the same example you quoted above - > spout a -> bolt b (emits 10 tuples per msg) -> bolt c > > I see the process latency and execute latencies under 5 ms both for B and > C. While the spout is at 1500ms. The Bolts dont don anything much other > than appending to an existing string. From what I understand, the complete > latency at a spout level is the time spent from nextTuple() to the time > ack() is called (if successful) and does not include the time the message > is spent waiting because of the Max Spout Topology property. To add to the > mysterry, i set the messasge timeout is at 1 sec. I dont see any failures > (fail() not called) but the spout latency is at 1.5 seconds. > > Regards, > Kashyap > > On Wed, Jul 29, 2015 at 10:35 AM, Nathan Leung <[email protected]> wrote: > >> No. You need to consider your system more carefully. As a trivial >> example, imagine you have spout a -> bolt b -> bolt c, with bolt b >> splitting tuple into 10 tuples. Each component has 1 task. If each >> component takes 1ms, your latency will not be the sum of the per bolt >> latency because of your fan out. >> On Jul 29, 2015 11:25 AM, "Kashyap Mhaisekar" <[email protected]> >> wrote: >> >>> Thanks Nathan. >>> >>> If I see the complete latency at spout is greater than the process >>> latencies of all bolts put together, does it mean that the ACKERS are a >>> problem and need to be increased? >>> >>> thanks >>> kashyap >>> >>> On Tue, Jul 28, 2015 at 7:30 PM, Nathan Leung <[email protected]> wrote: >>> >>>> The count is tracked from each spout task and does not include bolt fan >>>> out. If the setting is 100 and you have 8 spout tasks you can have 800 >>>> tuples from the spout in your system. >>>> On Jul 28, 2015 6:25 PM, "Kashyap Mhaisekar" <[email protected]> >>>> wrote: >>>> >>>>> Hi, >>>>> Does Max Spout Topology limitation apply to tuples emitted out of >>>>> bolts too? >>>>> For E.g.,. >>>>> 1. MAX_SPOUT_PENDING value is 1000 >>>>> 2. My Spout calls a bolt which emits 1000 tuples >>>>> >>>>> Does this mean there can be 1000X1000 tuples in the topology? Or does >>>>> it mean that only one tuple is emitted from Spout because each bolt emits >>>>> 1000 tuples? >>>>> >>>>> Thanks >>>>> kashyap >>>>> >>>> >>> >
