Trident has fully fault-tolerant, exactly-once processing *semantics*. Obviously if there's a failure it needs to retry messages, but it does it in such a way such that it's like messages were processed only once. Please see http://storm.apache.org/documentation/Trident-state.html for how this works.
On Tue, Dec 2, 2014 at 2:50 PM, Kushan Maskey < [email protected]> wrote: > But i thought trident method ensures you read the message only once. while > in traditional kafkaSpout reads the message until it is processed > successfully. Am I missing something here? Help clarify this. Thanks. > > -- > Kushan Maskey > 817.403.7500 > M. Miller & Associates <http://mmillerassociates.com/> > [email protected] > > On Tue, Dec 2, 2014 at 4:24 PM, Parth Brahmbhatt < > [email protected]> wrote: > >> The message is always replayed from the Spout, you will have to add dedup >> mechanism in B1 if you want to avoid duplicate processing. >> >> Implementing trident still won’t help you for the use case you are trying >> to solve. If you want intermediate bolts to dedup, you have to write the >> dedup logic. >> >> Thanks >> Parth >> >> On Dec 2, 2014, at 2:19 PM, Kushan Maskey < >> [email protected]> wrote: >> >> I have a main bolt B1 listening to spout S1. Once processing is done on >> B1 it emits message to B2, B3, B4 ad B5. After B5 process the data, it then >> emits this message to B6, B7and B8. In the process if B3 fails to process >> the data, then does the failed acknowledgement gets acked all the way back >> to the spout? Or if B3 fails, the message will just get re-emitted from >> bolt B1? >> >> In this case the message it going to get read twice and the data is >> duplicated. How do I prevent this to happen. We are using KafkaSpout that >> came in with storm.0.9.2. Is implementing trident method gonna help resolve >> this issue? Thanks. >> >> -- >> Kushan Maskey >> 817.403.7500 >> M. Miller & Associates <http://mmillerassociates.com/> >> [email protected] >> >> >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. > > > -- Twitter: @nathanmarz http://nathanmarz.com
