Assuming each flush period is associated with a start and end event, one approach is to instantiate a HashSet in the spout at flush start event. Each emitted tuple is added to the set, and upon receiving ack, it is removed from the set. Upon receiving flush end event, wait for all pending messages to be acked.
Assuming that processing latency is small, the pending set size should be relatively bounded in steady state at any point of time. When the HashSet size becomes 0, returns success. Of course, if flush periods are back-to-back, then start of the next flush period is same as the end of previous period. Thanks, Satish On Wed, Jun 24, 2015 at 4:20 PM, Javier Gonzalez <[email protected]> wrote: > Hi Satish, > > Thank you for your response. > > The ack at the spout guarantees that the flush message has traversed the > topology, but makes no guarantees about the tuples emitted before the flush > tuple. It would be possible for tuples to be in flight and be "beaten to > the ack" by the flush tuple, particularly if the flush tuple does not > participate in all the processing done in the bolts to the regular tuples. > > Regards, > Javier > On Jun 24, 2015 2:55 AM, "Satish Mittal" <[email protected]> wrote: > >> From the way Storm's acking mechanism works, if a spout tuple gets acked, >> then it implies that tuple has completely traversed the topology. Isn't >> that sufficient? >> >> On Tue, Jun 23, 2015 at 10:50 PM, Javier Gonzalez <[email protected]> >> wrote: >> >>> Hi, >>> >>> Question: how would you implement a "flush" in a topology: sending a >>> special message to the topology that will in time return with a message >>> that says everything up to the flush message has finished traversing the >>> topology? (does that make sense?) >>> >>> Regards, >>> Javier >>> >> >> >> _____________________________________________________________ >> The information contained in this communication is intended solely for >> the use of the individual or entity to whom it is addressed and others >> authorized to receive it. It may contain confidential or legally privileged >> information. If you are not the intended recipient you are hereby notified >> that any disclosure, copying, distribution or taking any action in reliance >> on the contents of this information is strictly prohibited and may be >> unlawful. If you have received this communication in error, please notify >> us immediately by responding to this email and then delete it from your >> system. The firm is neither liable for the proper and complete transmission >> of the information contained in this communication nor for any delay in its >> receipt. > > -- _____________________________________________________________ The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
