I just want to add, that duplicates in the result are easily possible. Let's say, the last bolt (ie, a sink), writes some tuples to a file before acking it. If the write is successful and the bolt fails for some reason before the ack goes through, the tuple will be replayed and written to the file a second time.
-Matthias On 05/18/2015 07:44 AM, Jake K. Dodd wrote: > 'At least once' partially (in some cases, entirely) depends on your > spout implementation. > > With the following topology: > > Spout --> Bolt A --> Bolt B --> Bolt C > > assume that Spout is Storm's officially-maintained Kafka spout. If Spout > emits a tuple, and subsequently that tuple times out during the > execution of Bolt C, Spout will replay the tuple--from the beginning of > the topology. Bolts A & B will then process the tuple a second time. > Thus, 'at least once.' Equivalently, if Bolt B explicitly fails the > tuple, then Spout will replay the tuple and Bolt A will process the > tuple a second time. > > If the operations in Bolt A,B are not idempotent, you might be in > trouble if Bolt C fails the tuple. > > Alternatively, you might have a custom Spout that silently ignores a > failed tuple, in which case Storm will not provide you an 'exactly once' > guarantee; it's up to the Spout to replay a tuple (or not) when it's > notified of a failed tuple tree. > > I don't know of any scenarios wherein a tuple will successfully traverse > the entire tuple tree and then be replayed by the spout. If there is > such a case, I'd like to hear about it. > > Bottom line is that if you're unable to make your bolts idempotent, and > this concerns you, look at Trident to get exactly-once guarantees. There > are all sorts of situations whereby you will encounter failed tuples, > some of which are entirely out of your control (e.g. network failures). > > Cheers > > Jake K Dodd > > On May 17, 2015, at 22:15, prasad ch <[email protected] > <mailto:[email protected]>> wrote: > >> Hi, >> >> In storm document they mentioned ,storm will process tuples at least >> once ,means it may be chance of processing same tuple more than once! >> >> >> *Can you explain what are the possible scenarios storm will Process >> more than Once?* >> >> >> am successfully setup storm cluster, here am using centOs my linux >> machines contains python 2.6 and daemontools inbuilt. >> >> >> >> Here am failed to start storm cluster setup under *daemon tools* >> ( http://cr.yp.to/daemontools.html ) >> ,*please help me* how to write script and hoe to start under it. >> >> >> >> >> >> >> >> >> Thanks! >> >> >> Reagrd's >> >> Prasad.ch <http://Prasad.ch>
signature.asc
Description: OpenPGP digital signature
