The message is always replayed from the Spout, you will have to add dedup mechanism in B1 if you want to avoid duplicate processing.
Implementing trident still won’t help you for the use case you are trying to solve. If you want intermediate bolts to dedup, you have to write the dedup logic. Thanks Parth On Dec 2, 2014, at 2:19 PM, Kushan Maskey <[email protected]> wrote: > I have a main bolt B1 listening to spout S1. Once processing is done on B1 it > emits message to B2, B3, B4 ad B5. After B5 process the data, it then emits > this message to B6, B7and B8. In the process if B3 fails to process the data, > then does the failed acknowledgement gets acked all the way back to the > spout? Or if B3 fails, the message will just get re-emitted from bolt B1? > > In this case the message it going to get read twice and the data is > duplicated. How do I prevent this to happen. We are using KafkaSpout that > came in with storm.0.9.2. Is implementing trident method gonna help resolve > this issue? Thanks. > > -- > Kushan Maskey > 817.403.7500 > M. Miller & Associates > [email protected] -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
