Re: [DISCUSS] Provision for dead-letter topic in storm

Yes good idea, would love to have this functionality of passing some user defined data from bolt to spout on failures. Ravi On 27 Sep 2016 10:05 p.m., "Kyle Nusbaum" wrote: > It seems to me that this can be solved by allowing a user to attach some > arbitrary

Re: [DISCUSS] Provision for dead-letter topic in storm

It seems to me that this can be solved by allowing a user to attach some arbitrary data to a call to fail(), which is passed to the spout. So there would be an override for fail in IOutputCollector which takes both the Tuple input and also some object to give to the spout. The spout's fail

Re: [DISCUSS] Provision for dead-letter topic in storm

Any more thoughts on this? Seems like a useful feature for all the spouts/bolts. On Wed, Sep 21, 2016 at 9:09 AM, S G wrote: > Thank you Aaron. > > We use Kafka and JMS spouts and several bolts - Elastic-Search, Solr, > Cassandra, Couchbase and HDFS in different

Re: [DISCUSS] Provision for dead-letter topic in storm

Thank you Aaron. We use Kafka and JMS spouts and several bolts - Elastic-Search, Solr, Cassandra, Couchbase and HDFS in different scenarios and need to have the dead letter functionality for almost all these scenarios. Locally we have this functionality almost ready for writing dead-letters to

Re: [DISCUSS] Provision for dead-letter topic in storm

I like the idea, especially if it can be implemented as generically as possible. Ideally we could "dead letter" both the original tuple and the tuple that itself failed. Intervening transformations could have changed the original tuple. I realize that's adds a lot of complexity to your idea and

[DISCUSS] Provision for dead-letter topic in storm

Hi, I want to gather some thoughts on a suggestion to provide a dead-letter functionality common to all spouts/bolts. Currently, if any spout / bolt reports a failure, it is retried by the spout. For a single bolt-failure in a large ADG, this retry logic can cause several perfectly successful