Hi Rao, One typical pattern for reporting non-retry-able errors / data in a non human-facing application is a dead letter queue, i.e. a queue somewhere that is plugged to an error monitoring tool. Most companies have already something like that in place (e.g. Nagios), so usually the idea is to get in concat with the person responsible for that system and understand how he wants us to send failure report to that existing system.
Another simple and efficient way to report an error is really just to log it, maybe with a separate logger instance that outputs all failure to a separate file/queue/db/whatever. In any case, the reporting itself is always easy, the sometimes more tricky part is to make sure your report's grab some human's attention so that he can take action. On Fri, Mar 21, 2014 at 8:09 AM, M Rao <[email protected]>wrote: > Hi Svend, > > > ---you need another specific error reporting tool, storm does not provide > such thing (atm at least, I don't know about the future). > > Can you please suggest any error reporting tool for us or any pointer or > link. It will be great help for us. > > Regards > Tarkeshwar > > > On 03/20/2014 04:01 PM, Svend Vanderveken wrote: > > HI Rao, > > > AFAIK there is no way to do that, and there is actually no master > controller node. > > Errors are reported to Storm whenever we want to trigger the Storm error > handling / exactly once semantic => the first point of decision is at the > place where the error occurs (or in a wrapper of your components): > > * if it makes sense to retry later (e.g. DB connection lost): throw a > FailedException to Storm > * otherwise (invalid tuple data), don't report anything to Storm > > The error reported to Storm is going to be propagated back to the > originating node where the corresponding spout is running. Bare in mind > that this is a very different concept than a master controller node since > we typically have plenty of spout instances: e.g. in case of Kafka, if we > have 100 Kafka nodes each with 10 partitions, we are able to start up to > 1000 instances of Storm Kafka spouts on plenty of hosts. The spout is going > to handle the replay mechanism of that tuple, according to the > transactional/opaque semantic that it implements. > > If you need to do error reporting for other reasons that the Storm > replay mechanism (typically reporting for at least the tuples you decide > not to retry, so you can investigate them later without blocking the > real-time flow of events), then you need another specific error reporting > tool, storm does not provide such thing (atm at least, I don't know about > the future). > > Best regards, > > Svend > http://svendvanderveken.wordpress.com/ > > > > > > > > > > > > > > > > > > > > On Thu, Mar 20, 2014 at 5:23 AM, M Tarkeshwar Rao < > [email protected]> wrote: > > Hi, > > > > Can you please help me in finding the strategies for fault tolerant in > (Trident)storm. > > I want to properly send the failed reason to Master controller node. > > > > Regards > > Tarkeshwar > > > > > > >
