@Sam. It's not so simple to recognize if it's a network outage or it's slowly running insert to db (in the storm core) V
On Wed, Nov 5, 2014 at 9:38 PM, Sam Mati <[email protected]> wrote: > This is intended behavior of Storm (which I strongly disagree with). > "timeouts" are intended to fail any tuples that have been lost due to > workers going down, network issues, etc… not as a means of controlling flow. > > You should set max pending lower, set the timeout higher, and/or have > your Bolts ignore "old" tuples by adding a timestamp to the original tuple > and having the "execute" method of Bolts test that the tuple is not too old. > > Other threads that address this: > "KafkaSpout stops pulling data after a few hours" > "What is the purpose of timing out tuples?" > > Best, > -Sam > > From: Nilesh Chhapru <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Wednesday, November 5, 2014 1:13 PM > To: "[email protected]" <[email protected]> > Subject: Message Flows, Even After Timeout > > Hi All, > > > > I have a (Storm Kafka Spout) Kafka spout which emit to a bolts, post some > computation over the message it is passed back to other Kafka topic using a > Kafka Producer. > > > > In my application even if the message is gets timeout it still loads the > message to the producer, which results duplicate entries in the resulting > spout. > > > > Is there a way to kill the message if it times out, as for me storm isn’t > killing the message even when the spout failed method is called. > > > > Kindly let know if I am missing out some configuration here. > > > > *Regards*, > > *Nilesh Chhapru.* > > ------------------------------ > > ---------------------------------------------------------------------------------------Disclaimer---------------------------------------------------------------------------------------------- > > ****Opinions expressed in this e-mail are those of the author and do not > necessarily represent those of Ugam. Ugam does not accept any > responsibility or liability for it. This e-mail message may contain > proprietary, confidential or legally privileged information for the sole > use of the person or entity to whom this message was originally addressed. > Any review, re-transmission, dissemination or other use of or taking of any > action in reliance upon this information by persons or entities other than > the intended recipient is prohibited. If you have received this e-mail in > error, please delete it and all attachments from any servers, hard drives > or any other media. > > Warning: Sufficient measures have been taken to scan any presence of > viruses however the recipient should check this email and any attachments > for the presence of viruses. Ugam accepts no liability for any damage > caused by any virus transmitted by this email. **** >
